Papers with robust framework
EdTec-QBuilder: A Semantic Retrieval Tool for Assembling Vocational Training Exams in German Language (2024.naacl-demo)
Copied to clipboard
| Challenge: | Existing methods to gather test items from validated item databases are under-researched, but there is little research on assembling exam items from a database of valid items. |
| Approach: | They propose to use semantic search to assist vocational educators in assembling exam forms by using eight retrieval strategies and 25 popular sentence similarity models. |
| Outcome: | The proposed tool is based on eight retrieval strategies and 25 popular pre-trained sentence similarity models. |
Improving and Assessing the Fidelity of Large Language Models Alignment to Online Communities (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown promise in representing individuals and communities, but evaluating their fidelity remains a challenge. |
| Approach: | They propose a framework for aligning large language models with online communities via instruction-tuning and comprehensively evaluating alignment across various aspects of language. |
| Outcome: | The proposed framework shows that it can be used to create high-fidelity representations of people and communities. |
ToVo: Toxicity Taxonomy via Voting (2025.findings-naacl)
Copied to clipboard
Tinh Son Luong, Thanh-Thien Le, Thang Viet Doan, Linh Ngo Van, Thien Huu Nguyen, Nguyen Thi Ngoc Diep
| Challenge: | Existing toxic content detection models face limitations due to the closed-source nature of training data and the paucity of explanations for their evaluation mechanism. |
| Approach: | They propose a mechanism that integrates voting and chain-of-thought processes to produce a high-quality open-source dataset for toxic content detection. |
| Outcome: | The proposed model improves transparency and customizability while facilitating better fine-tuning for specific use cases. |
Fully Statistical Neural Belief Tracking (P18-2)
Copied to clipboard
| Challenge: | Existing framework for a dialogue state tracking model requires an expensive manual retuning step . |
| Approach: | They propose to improve existing NBT model by removing a manual retuning step . they propose two different statistical update mechanisms to improve model performance . |
| Outcome: | The proposed model achieves competitive performance and provides a robust framework for building resource-light DST models. |
ReAct Meets Industrial IoT: Language Agents for Data Access (2025.emnlp-industry)
Copied to clipboard
| Challenge: | a framework for domain-specific language agents is being developed for industrial automation . a novel approach to adapting these systems to domain-based applications poses new challenges . |
| Approach: | They propose a framework for deploying domain-specific language agents that can query industrial sensor data using natural language. |
| Outcome: | The proposed framework outperforms standard prompting baselines across multiple LLMs including smaller models. |
Learning Unsupervised Multilingual Word Embeddings with Incremental Multilingual Hubs (N19-1)
Copied to clipboard
| Challenge: | Recent research has found that a shared bilingual word embedding space can be induced by projecting monolingual word embeds from two languages without any bilingual supervision. |
| Approach: | They propose a framework for learning unsupervised multilingual word embeddings that mitigates instability issues for distant language pairs. |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on two downstream tasks outperforming even supervised baselines. |
BacktrackAgent: Enhancing GUI Agent with Error Detection and Backtracking Mechanism (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing GUI agents focus on enhancing the accuracy of individual actions and lack effective mechanisms for detecting and recovering from errors. |
| Approach: | They propose a framework that incorporates a backtracking mechanism to enhance the task completion capabilities of GUI agents by verifier, judger, and reflector components. |
| Outcome: | The proposed framework improves task success rate and step accuracy on Mobile3M and Auto-UI benchmarks. |
Automated Progressive Red Teaming (2025.coling-main)
Copied to clipboard
| Challenge: | Automated red teaming (ART) is effective but time-consuming, costly and lacks scalability. |
| Approach: | They propose an automated red teaming framework that generates adversarial prompts to expose LLM vulnerabilities. |
| Outcome: | The proposed framework explores and exploits LLM vulnerabilities through multi-round interactions. |
UCFE: A User-Centric Financial Expertise Benchmark for Large Language Models (2025.findings-naacl)
Copied to clipboard
Yuzhe Yang, Yifei Zhang, Yan Hu, Yilin Guo, Ruoli Gan, Yueru He, Mingcong Lei, Xiao Zhang, Haining Wang, Qianqian Xie, Jimin Huang, Honghai Yu, Benyou Wang
| Challenge: | Recent advances in large language models (LLMs) have expanded their potential applications in finance. |
| Approach: | They propose a framework to evaluate the ability of large language models to handle financial tasks using human expert evaluations and task-specific interactions. |
| Outcome: | The proposed framework evaluates the ability of large language models to handle complex financial tasks and combines human expert evaluations with dynamic, task-specific interactions to simulate the complexities of evolving financial scenarios. |
Query4Regex: Verifiable Regex Transformation through Formal Operations from NL and DSL Queries (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) generate structured data, but their ability to precisely manipulate it remains relatively under-explored. |
| Approach: | They propose a benchmark to evaluate verifiable transformations on regexes . they use natural language instructions and a program-like domain-specific language that specifies the sequence of operations to evaluate LLMs. |
| Outcome: | The proposed benchmark compares LLM performance on natural language and DSL queries for regex manipulation. |
Towards Automatic Evaluation for Image Transcreation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for evaluating image transcreation have relied on human evaluation. |
| Approach: | They propose a suite of automatic evaluation metrics inspired by machine translation metrics . they identify cultural relevance, semantic equivalence and visual similarity as critical dimensions of image transcreation . |
| Outcome: | The proposed evaluation metrics agree with human ratings across 7 countries. |
HARE: an entity and relation centric evaluation framework for histopathology reports (2025.findings-emnlp)
Copied to clipboard
| Challenge: | evaluating the clinical quality of medical domain automated text generation remains a challenge. |
| Approach: | They propose a framework for histopathology automated report evaluation that prioritizes clinically relevant content by aligning critical histo pathology entities and relations between reference and generated reports. |
| Outcome: | The proposed framework outperforms existing metrics in histopathology report evaluations. |
From 128K to 4M: Efficient Training of Ultra-Long Context Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Long-context capabilities are essential for document and video understanding, in-contact learning, and inference-time scaling. |
| Approach: | They propose an efficient training recipe for building ultra-long context LLMs from aligned instruct model, pushing the boundaries of context lengths from 128K to 1M, 2M, and 4M tokens. |
| Outcome: | The proposed model extends the context window while maintaining short context capabilities while maintaining the performance of the existing model. |
Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge (2025.findings-emnlp)
Copied to clipboard
Yan-Lun Chen, Yi-Ru Wei, Chia-Yi Hsu, Chia-Mu Yu, Chun-Ying Huang, Ying-Dar Lin, Yu-Sung Wu, Wei-Bin Lee
| Challenge: | Large language models (LLMs) demonstrate strong task-specific capabilities through fine-tuning, but merging multiple fine- tuned models often leads to degraded performance due to overlapping instruction-following components. |
| Approach: | They propose a layer-wise approach that assigns layer-specific weights to task vectors based on their alignment with instruction-following or task-specific components. |
| Outcome: | The proposed approach outperforms existing methods in learning and forgetting tasks while preserving overall model utility. |
Graphically Speaking: Unmasking Abuse in Social Media with Conversation Insights (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to detect abusive language often ignore conversational context, leading to inconsistent and sometimes inconclusive results. |
| Approach: | They propose a graph neural network approach that uses conversational context to model social media conversations as graphs, where nodes represent comments and edges capture reply structures. |
| Outcome: | The proposed model outperforms baseline and linear context-aware methods and achieves significant improvements in F1 scores. |
VISIAR: Empower MLLM for Visual Story Ideation (2025.findings-acl)
Copied to clipboard
Zhaoyang Xia, Somdeb Sarkhel, Mehrab Tanjim, Stefano Petrangeli, Ishita Dasgupta, Yuxiao Chen, Jinxuan Xu, Di Liu, Saayan Mitra, Dimitris N. Metaxas
| Challenge: | Existing literature on visual storytelling has not explored the ideation process fully. |
| Approach: | They propose a visual story ideation task that automates the selection and arrangement of visual assets into coherent sequences that convey expressive storylines. |
| Outcome: | The proposed framework surpasses baseline by 33.5% and 18.5%, respectively, on three metrics. |
Exploring Intra and Inter-language Consistency in Embeddings with ICA (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that ICA can reveal universal semantic axes across languages but lack verification of consistency of independent components within and across languages. |
| Approach: | They propose to use independent component analysis to identify independent components that are more interpretable than PCA to find universal semantic axes. |
| Outcome: | The proposed framework ensures the reliability and universality of semantic axes. |
HAG: Hierarchical Demographic Tree-based Agent Generation for Topic-Adaptive Simulation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to Agent-Based Modeling fail to adapt to unseen topics absent from data. |
| Approach: | They propose a Hierarchical Agent Generation framework that formalizes population generation as a two-stage decision process. |
| Outcome: | The proposed framework outperforms baseline models in a multi-domain benchmark and comprehensive evaluation framework. |
Beyond Single View: A Comprehensive Benchmark for Medical Multimodal Large Language Models on Multi-Image Understanding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for multimodal large language models are limited to multiview diagnostics. |
| Approach: | They propose a benchmark specifically designed for medical multi-image understanding that evaluates MLLMs across four dimensions. |
| Outcome: | The proposed model performs better in multi-image contexts than open-source models . the model perform better when processing increased visual loads than closed-source ones . |
MemeDetoxNet: Balancing Toxicity Reduction and Context Preservation (2025.findings-acl)
Copied to clipboard
| Challenge: | Toxic memes spread harmful and offensive content and pose a significant challenge in online environments. |
| Approach: | They propose a framework to mitigate toxicity in toxic memes by leveraging a set of pre-trained models that can interpret the visual and textual components of memes. |
| Outcome: | The proposed framework reduces toxicity on publicly available meme datasets by 10-20% compared to the previous methods. |
Towards semantic reliable clinical QA: Query pipeline optimization for cancer patient question answering systems (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are promising for medical Question-Answering but suffer from hallucinations that jeopardize patient safety. |
| Approach: | They propose a three-level controllable metadata-aware framework optimized for Cancer Patient QA (CPQA) they propose combining semantic retrieval with real-time Boolean search to overcome metadata blindness. |
| Outcome: | The proposed framework improves the answer accuracy of Claude-3-haiku by 5.24% over chain-of-thought prompting and about 3% over a naive RAG setup. |
TinySQL: A Progressive Text-to-SQL Dataset for Mechanistic Interpretability Research (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing text-to-SQL datasets are too complex and noisy for rigorous interpretability analysis. |
| Approach: | They propose text-to-SQL generation as an ideal task to study mechanistic interpretability . they use edge attribution patching and sparse autoencoders to identify minimal circuits . |
| Outcome: | The proposed task combines the formal structure of toy tasks with real-world complexity. |
Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules (2026.findings-acl)
Copied to clipboard
| Challenge: | *SchED* is a training-free, model-agnostic early-exit algorithm that terminates diffusion decoding using a progress-aware confidence threshold. |
| Approach: | They propose a training-free, model-agnostic early-exit algorithm that terminates diffusion decoding using a progress-aware confidence threshold. |
| Outcome: | The proposed algorithm achieves 4 speedups on instruction-tuned models while maintaining baseline performance on average. |
TRAC: Teacher-Guided Token Reward with Adaptive Calibration for Robust Policy Optimization (2026.acl-long)
Copied to clipboard
| Challenge: | Current reward models for reinforcement learning (RL) rely on outcome rewards that propagate a single scalar value across all tokens based on final correctness. |
| Approach: | They propose a framework that derives dense token-level supervision from LLMs . they use a multi-granularity calibration mechanism to modulate teacher influence . |
| Outcome: | The proposed framework evaluates teacher reliability across problem-level expertise, trajectory-level discrimination, and token-level confidence. |